Heuristic Evaluation

Interface BESS project prototype: SIMS Textbook Exchange
Neat interface. Bright and fresh color scheme. Keep going!

..................................................................................................................

Severity [number of violations]

Severity 3 [15]
Severity 2 [10]
Severity 1 [05]
Severity 0 [02]

Missing Information
Missing Functionality
Under Development?
General Problems
General Suggestion

Appendices

Severity Rating Scheme
The Heuristic Evaluation Guidelines
Evaluation of SFnight by SIMS Textbook Exchange Team

...................................................................................................................

SEVERITY 3 [15]

1. [H1: Visibility of system status ] (Severity 3)
There is no information about how many books are available for sale. The user cannot gage how large the forsale book pool is. Also, the user cannot tell if the site is for searching SIMS textbooks only or also includes related books and readers?

2. [H2: Match between system and the real world] (Severity 3)
When completing Task 1 - looking for "buy" books. The tabs "sell books" and "edit posting" seem pretty obvious as for what they may do. The not so obvious tab / link choice is "find books". Not sure if "find books" relates to buy since search is considered a separate task.

Possible Solution(s)
If I recall, the concern with "buy" was that it isn't guaranteed users will be able to buy something, but neither is "sell". The words communicate the action that can be completed even if the site does not guarantee that users will be able to complete either goal. A separate disclaimer about the services offered may be a better solution.

A possible compromise design can just change the description link in the homepage to "find books to buy" - get that notion into the user head and then perhaps find book tab is understood to also allow "buy".

Perhaps keep the "find books" tab to use for search purposes. See next comment about this. (This also conveniently tells a student all the books needed for a course.) And then add a "buy books" tab that simply lists what is available for sale. See Missing Functionality. Users can see at a glance what is available. I don't think there will be that many books that the page will be ridiculously long.

3. [H2: Match Between System and the Real World] (Severity: 3)
In Sell Books page, after user completes the posting steps, the new page will show a link for 'View your new posting as it appears on BESS'. Following that link, the user will see all the other people who are also selling that book, and make an offer to buy! I can think of no task scenario for a user to buy and sell the same book at same time, so this function is out of place.

Possible Solution(s)
Provide a preview mode in which users can go back to editting the posting and/or change the sentence to "Check your posting now".

4. [H3: User control and freedom ] (Severity 3)
When selling a book, user cannot add new book to sell. She can only choose from one of the existing books. Ideally, the systems should give user the flexibility and control of what book she want to sell. Given the limitations of current project scope, it should be clear to the users from the beginning the type of books the system provides.

5. [H3: User control and freedom] (Severity 3)
By clicking at Amazon.com at the Find Books, users may navigate there and get lost to go back to the system. Better provide the link to another window.

6. [H4: Consistency and standards] (Severity 3)
The interface mixes buttons with sentences, like in Buy a Book page. This is the result of the problem related to Minimalism Design and use of too much information.

Possible Solution(s) Instead of "These people are selling this book:", I would recommend something like "Choose the Book Sellers You are Interested to contact to Make an Offer:" and after the listing, only the button "Make an Offer" and "Cancel" buttons

7. [H4: Consistency and standards] (Severity 3)
The interface presented an E-mail address layout not standard, and not helpful for those users who automatically copy and paste their e-mail, or very used to quickly type.

8. [H4: Consistency and standards] (Severity 3)
It is not clear why there should be a login link as well as an account button. The two should really mean the same, an area where the user needs to login in order to perform tasks with his/her account.

Possible Solution(s) Keep either the login link or account button, not both.

9. [H5: Error prevention] (Severity 3)
In the Login page, the user is presented with both user login for existing members and sign up for new members. The current layout requires users to choose whether they are new or existing users, and then continue. This can be confusing. Simply make it one step for new member sign up, and a separate group of steps to complete the login process by existing members. Also for the existing member login area, instead of saying "email address (your login...)", use the standard of "login (your email address)".

10. [H5: Error prevention] (Severity 3)
The Search feature presented in the BESS interface can confuse users, and generate an error message. The "Go!" button looks as if it is only related to the "Title". Since there are only a few categories, an appropriate cluster layout using radio buttons can display all the categories and prevent errors. Dropdown menu is more approparite for a much bigger options listing. A non default mark is also recommended. By having the initial sellection in Title, user might forget after providing the keyword. Here's a suggested design:

SEARCH By: title author ISBN

11. [H6: Recognition rather than recall] (Severity 3)
In the search results, users do not know if the book listed is actually available for purchase. One has to actually look up each individual book description to find out if there are copies available for sale.

Possible Solution(s)
Simply indicate the number of copies available for sale next to each book.

12. [H6: Recognition Rather than Recall] (Severity: 3)
In Sell Books page, user needs to enter personal information every time s/he wants to sell books out. Could be a tedious process if a second grade student wants to sell out a big part of inventory

13. [H8: Aesthetic and minimalist design] (Severity 3)
The interface has too many explanations that we feel are not needed. A general problem is the extra textual description that accompanies processes. The extra units of information compete with user's goals of saving time by forcing users to do extra reading. This is found throughout the whole interface.

An Example
In the make offer page, the reminder by the "Send offer" button: "Remember - there's no obligation to buy", feels out of place. By the time the user is ready to click the button, the user is ready to complete the task and the reminder just makes the user think again - as if questioning the user's decision at that point, thus somewhat annoying.

Possible Solution(s)
I think the response page does a good job of reminding users of the no obligation policy. To address any doubts during the process of filling out the form and before clicking on the "Send offer" button, there should be a way, say a link to a policy page, that allows the user to check the policy on making offers before doing so.

Another Example
At Account Maintenance: It would be enough to say something like:

"Here you can:

Edit your postings
Change your password
Delete your BESS account "

In general, if there is a button, their personas would know that the buttons are to be pressed. That if they need more information they will go to Help. The interface should use less words and make the process itself easy. Consequently all the sentence "Choose one of the buttons…page" is unnecessary.

14. [H10: Help and Documentation] (Severity: 3)
There is a Login hotlink in the upper right area of each page (it turns Logout in Sell page). But how login can benefit the user is not mentioned in the pages, no even an explanation for its functionality. Clicking on Help, it gives less than under Account.

15. [H11: Thrustworthiness] (Severity 3)
Trust became an issue in web systems. Users need to know who is offering the system and who is responsible for its maintenance. This information probably will be available at the Member Guidelines and Privacy Policy. But at the homepage something minimal such as the identity of the responsible organization should be visible, so users would know who is responsible for the system. Also users need to know how to contact the responsible organization/individual in case of problems.

^

SEVERITY 2 [10]

1. [H1: Visibility of System Status] (Severity: 2)
After user/seller edited the previous posting, the new page would not show the title(s) affected. Feedback to user is needed.

2. [H2: Match between system and the real world] (Severity 2)
This is a case where the system duplicates the real world so much that the problem of the real world is also duplicated. There are many foreigne SIMS students that may not know the meaning of Obo/Firm. When one is trying to sell a book, the term "firm" may become obvious from the choices available. But the "willing to bargain" still does not translate to "obo" even if one already knows what that means.

Possible Solution(s)
We think the terms should still be used since a user only need to learn the meanings once. Since this is a popular word on auctions, maybe a link to a explanation at help would provide a better understanding. In the sell area, could put "obo" in parenthesis next to "willing to bargain".

3. [H3: User control and freedom] (Severity: 2)
In Sell Books page, user cannot use Shift- key or Ctrl- key to select multiple books at the same time, which is already a common practice in web page.

4. [H4: Consistency and standards] (Severity 2)
The interface used the link Cancel and the button Cancel in different pages. Users may be confused by this different design for the same function.

Possible Solution(s)
It is reccomended to use the Cancel button.

5. [H4 Consistency and Standards] (Severity: 2)
In Find Books, clicking SIMS 202 will lead user back to home page. Clicking on any other class will let user stay in Find Book area

6. [H5: Error prevention] (Severity 2)
At Create A New Account, the expression "Repeat password" doesn't give the user a better sense why he/she should repeat the password, specially those who are in a hurry.

Possible Solution(s)
Change to "Confirm password": this sentence by itself give the user the idea that is necessary to confirm it.

7. [H7: Flexibility and efficiency of use] (Severity 2)
Find book interface is not very flexible and efficient. Although user may browse available books course by course, he/she may also want to view a complete list of available books. There is no way for him/her to do this.

Possible Solution(s)
A possible solution is to have a list of all books sorted alphabetically by title and let user to do multiple choice by checkbox. In this way, user can view details of several books he/she may be interested.

8. [H8: Aesthetic and minimalist design ] (Severity 2)
Testimonials can be a minor distraction for user, since it gives information unrelated to the task.

Possible Solution(s)
Something related to course or the book may be better.

9. [H8: Aesthetic and minimalist design] (Severity 2)
The link to Home at Find Books for Sale is unnecessary. User has a clear option to click the button Home.

10. [H10: Help and documentation] (Severity 2)
Help and documentation are centralized in a separate page in this system, which is not convenient for users. Additional help in form of brief instruction and tips might be also available aside each interaction point. User can get instant access to help when she is interacting with the system, instead of interrupts her task for going to help section. (See general comment about help.)

^

SEVERITY 1

1. [H1: Visibility of system status ] (Severity 1)
While Edit Posting page shows the cover picture of the book that user wants to sell, the Posting page does not show it, which would give to user a comfirmation of posting the right book and edition.

2. [H4: Consistency and standards] (Severity 1)
The interface presents all command buttons (such as Cancel, Continue etc) at the left side. The right side is not only a standard (at the Occidental world, at least), but also directs the eye to the end of a step or process.

3. [H4: Consistency and standards] (Severity 1)
The system provide two directions at the Login page regarding login. The first direction, "Please Login!" makes users think it's critical to login. The second direction, "You need to login if you want to post a book for sale", makes login not so critical. "Please login" only would make sense if it would be mandatory to use the system.

4. [H4: Consistency and standards] (Severity 1)
The interface presents a feature called "Quick Search for Books", which makes user expect a more advanced search (The Find one?), as it is usually presented in the software and web environment.

Possible Solution(s)
"Quick search" is usually used for keyword searching. Just using "Search for Books" will make it more clear about the features offered. Also the use of the word "Quick search" to represent both browse and search functions is not reccomended .

Note: a new design was presented for Quick Search for Books at Create My Account, separating the Quick search from the Browsing function. But since this was not consistent with the other pages layout, I think their intention was to present as appears at the Home page.

5. [H7: Flexibility and efficiency of use] (Severity 1)
The interface presents price comparison at the page Buy a Book View . The personas may be more interested in seeing the lower prices first, such as presented in the same page at the sellers. Also aligning the results in the small table always accelerates the information scanning.

^

SEVERITY 0

1. [H7: Flexibility and efficiency of use] (Severity 0)
Information about edition and year is very important for the courses' requirements, and for price bargains. The system does not mention anything about the year, and presents uneditable Edition information at: Browse by Class results and Sell a Book form. Some students, such as the persona Sarah McClintock, might be willing to pay less if the edition is older. Can the sellers specify they have other edition instead? Or they only can say this in the Comments section? Which has the separate problem of not being available to buyers.

2. [H8:. Aesthetic and minimalist design] (Severity 0)
The homepage needs a better explanation/presentation about what the system does and how can it help students meet their goals. It also would help market the provided service by highlighting the features. Personas' main goals are to buy/sell textbooks. They 'get in touch' for this purpose, not to make friends from the start. The site facilitates communication for the primary goal.

Possible Solution(s)
A bullet presentation will help new users understand better what is the system for:

How BESS can help you to SELL and BUY textbooks:

SELL
You can post the books you want to sell to a large community of potential buyers.
SIMS students will find the books you would like to sell
Etc.

BUY
Easily find who is selling the books you need.
Compare prices between BESS sellers and Amazon.com, etc

DISCOVER
New ways of contacting SIMS students and make friends

^

MISSING INFORMATION

The interface has an optional field for Seller comment about the book, but it is not clear where the Comments will appear.

 

MISSING FUNCTIONALITY

The interface at Buy a Book View did not provide to users ability to sort the results by Date, Price, Conditions, etc. Considering that each semester there are 40 students, an unpopular book from a mandatory class may result in a very extensive seller listing (also considering that this system will be a success :). The personas may be interested in first seeing only the one with better prices, or the ones with better conditions, etc.

^

UNDER DEVELOPMENT? (We are not sure if the following should be evaluated as part of the prototype or still underdevelopment. In any case, issues to keep in mind for future development.)

[H5. Error prevention ] (Severity 3)
When user opens an account, the systems requires a password and confirmation. There is no check of consistency between password and confirmation. Nothing prevents user from entering inconsistent password.

GENERAL PROBLEMS (Problems/bugs probably due to limitations of the implementation and not real usability problems of the design.)

1. In the make offer page, the seller section do not reflect the actual seller(s) the user has chosen.
2. When using the Browse by Course feature in the Home page, selecting 210 brings the user to a different search interface than the others.

GENERAL SUGGESTION

We suggest getting rid of the term "help" period. Instead, have something like "About BESS" or "BESS FAQ". The area would include info about the limits of the system, policy, definitions, terms of use, what can be done with accounts, etc.

 

^

APPENDICES

Severity Rating Scheme 0. Don't think this is a usability problem. 1. Cosmetic problem 2. Minor usability problem 3. Major usability problem; important to fix 4. Usability catastrophe; imperative to fix

The Heuristics Evaluation Guidelines reproduced from Nielsen's list. 1. Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. 2. Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order. 3. User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo. 4. Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions. 5 .Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. 6. Recognition rather than recall Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. 7. Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions. 8. Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. 9. Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. 10. Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.